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ABSTRACT 

Between 1985 and 1988, the Northwest Regional 
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followed by a brief description of the CHIP workshop series. The 
results of a follow-up study in the winter of 1989 using qualitative 
data obtained from interviews with 35 participants from 33 school 
districts in Colorado, Wyoming, and New Mex^^o are then presented, 
along with an analysis of the findings and relevant illustrations. 
The three-session CHIP workshop program is based on literature 
associated with teacher effectivenec;s, school effectiveness, 
educational change, content area advances, research on Chapter l 
effectiveness, and exemplary Chapter 1 programs. The CHIP model 
coordinates federal and state law and regulations, parental 
participation, compensatory programs and instruction, regular class 
instruction, other school programs, district staff, community 
members r and student achievement and other student gains. (TJH) 
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Results of a Process for Improving Chapter 1 Programs 

Shelley H. Billig, Sam stringfield, and Alan Davis 

Prepared for the American Educational Research Association 
Annual Meeting, San Francisco, March, 1989 

Policymakers and practitioners should recognize that 
the improvement of the Chapter 1 programs will occur on 
a site-by-site basis. Few sweeping reforms will effect 
uniform changes in local instructional programs. 

The Design and Implementation o f Chapter 1 

Instructional Services 

(Rowan, Guthrie, Lee, and Guthrie, 1986) 

Between 1985 and 1988, the Northwest Regional Educational 
Laboratory chapter 1 Technical Assistance Center (NWREL-TAC) 
developed, refined and disseminated a research based process 
for improving local compensatory education programs. Known 
as the Chapter 1 Improvement Process (CHIP) , it combined 
knowledge from five research areas into a year-long, cross- 
_ district, highly interactive program improvement effort. 

(Estes, 1988.) 

The series has been offered in 16 states and attended by 
representatives from 374 LEAs, involving approximately 1100 
participants. Although at least two versions of the serxes 
have been offered, the themes and general approach in each 
is the same. 

Pilot studies of the effectiveness of the CHIP at the local 
level were presented at AERA by Davis (1988) and Leitner 
(1988). In the two presentations, a total of six projects 
from four districts were examined. The two studies 
concluded that, in general, participating projects: 

a) Repojcted considerable satisfaction with the workshop 
series, 

b) Were continuing to follow the process, post 
implementation , 

c) Repojrted increased satisfaction with the Chapter 1 
program, both on the part of the Chapter 1 staff, and 
the professional staff of the schools they served, 

d) In the three contexts with available outcome data, 
analyses supported the increased effectiveness (p < .Of 
in each instance) of the Chapter 1 program, post CHIP. 

This paper will examine the development and effects of the 
CHIP model in more depth. After a review of relevant 
literature, the development of the CHIP model will be 
discussed, followed by a brief description of the CHIP 
workshop series. The results of a follow-up study using 
qualitative data will then be presented, along /ith an 
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analysis of the findings with relevant illustrations. 
Finally, conclusions from this research and implications for 
both further research and for federally mandated Chapter l 
program improvement efforts will be described. 

DEVELOPMENT OF THE CHAPTER 1 IMPROVEMENT PROCESS 

The design of the Chapter 1 program improvement workshop 
series was based on findings from the research literatures 
in five fields: teacher effectiveness, school effectiveness, 
school/program change, advances within the content areas, 
research on Chapter 1 effectiveness and exemplary chapter 1 
programs. CHIP provides LEA participants with the 
opportunity to view this substantial body of research 
through local lenses. 

Teacher effectiveness resee^rch concerns the effects of 
teachers' in-class behaviors on aggregated measures of 
student achievement. Reviews of the research (Brophy and 
Good, 1986) reveal that teachers' in-class behavior patterns 
produced powerful, predictable effects on student 
achievement. Such variables as Academic Learning Time (ALT) 
(Fisher et al., 1980), Interactive Instruction (Stallings, 
1980) , directed use of praise (Brophy, 1981) , and 
questioning patterns used by teachers in classrooms can be 
altered to produce increased student learning. 

In the school effectiveness field, work by Edmonds (1979), 
Brookover et al. (1979), Barr and Dreeben (1983), Teddlie et 
al. (1984), Mortimore et al. (1987), and others (for reviews 
see Purkey and Smith, 1983; Stringfield and Teddlifi, 1988) 
indicate that larger institutions make a significant 
difference in such areas as "positive school and classroom 
climate," "clear goals and objectives," and the constructive 
use of evaluation data. Both the Mortimore et al. and 
Teddlie et al. studies found that highly effective schools 
which were serving disadvantaged youth obtained greater mean 
student achievement than did less effective schools serving 
middle class populations. 

Barr and Dreeben (1983), Mortimore et al. (1987), and' 
Stringfield and Teddlie (1988) nested teacher effectiveness 
measures within school effectiveness studies. Each study 
confirmed that in higher achieving schools, teachers were 
more likely to follow the instructional precepts suggested 
in the teacher effectiveness research. 

Studies of the processes of educational change in a variety 
of contexts such as those by Rosenblum and Seashore-Louis 
(1981), Fullan (1982), Huberman and Miles (1984), and Hall 
and Hord (1987) possess considerable congruence as to the 
conditions necessary for improvement. All found that 
meaningful change is a multi-year process, with primary 
roles for instructional leadership and technical assistance. 
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The value of commitment by those who must implement the 
change has been repeatedly supported, as have been the needs 
tor clear vision and goals, early success, sustained 
interaction and staff development efforts among and for 
involved staff. ^ 

Research advances within the content areas, especially in 
reading, have contributed to the body of knowledge 
surrounding learning and student achievement. Research on 
huraan cognition in the psychology of language, child 
development, environmental influences on learning, and 
classroom processes were all cited as significant influences 
on the production of Becoming a Nation of Readers (Anderson, 
Hiebert, Scott and Wilkinson, 1985.) Romberg (1986) cited a 
similar list regarding research contributions to the field 
of mathematics education. 

Wimpelberg (1987) and Fullan (1982, ch. 10) provided 
intellectual frameworks and research reviews for viewing 
programmatic change led by an LEA's central administrative 
staff. Davis (1988) came to similar conclusions from a 
Chapter i perspective. A cost-benefit analysis by Levin and 
Meister (1986) , a "best evidence synthesis" by Slavin 
(1987) , a whole-school vs traditional Chapter l comparative 
study by Stallings (1987), the Designs for Compensatory 
Education conference sponsored by OERI (Williams et al., 
1986) , two technical investigations by regional Chapter 1 
Technical Assistance Centers (Heppler et al., 1987; 
Stringfield et al., 1989), and the U.S. Secretary of 
Education's Exemplary Compensatory Education Recognition 
Project all point to the potential for significant program 
impact above simple teacher or school level effects. 

A Model of Chapter i Effects 

Using the above five research base , it is possible to 
construct a series of increasingly refined models of Chapter 
1 effects on student achievement and other desired outcomes. 
The simplest model, implicit in the most widely used federal 
Title 1/Chapter l Evaluation Reporting System (TIERS Model 
A) IS represented in Figure l. For 20 years, researchers 
have attempted to ascertain simple program effects on 
students through this model. Using Figure 1, researchers 
assume that Chapter i programs have a direct effect upon 
student outcomes with no other factors intervening, or all 
other factors cancelling each other's effects, in the 
relationship. In many regards, the Coleman Report (Coleman 
et al., 1956) employed a model of this sort, substituting 
"school" for "program." Using such a model, Coleman (at the 
school level) and Carter (at the Title 1 program level) 
found only modest evidence of school/program effects. 
Pupils do not attend Chapter 1 programs in a vacuum. 
Research by Rowan et al. (1988) indicates that typical 
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Chapter 1 students spend only 40 minutes of each 6 hour 
school day in compensatory education programs. 

RGURE 1. SIMPLEST COMPENSATORY EFFECTS MODEL 



PROGRAM 



STUDENT ACHIEVEMENT GAIN 

AND 0T^:5n desired outcomes 



The more recent school effectiveness studies (e.g., Barr and 
Dreeben, 1983; Mortimore et al. , 1987; Stringfield and 
Teddlie, 1988) have employed more complex models of 
schooling. This model, represented in Figure 2, reflects 
the reality that students spend their school days in one or 
more classrooms, and that each classroom is affected by 
events in other classrooms, by the ethos of the school and 
by the community. 



FIGURE 2: "XHOOL EFFECTS MODEL 
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The CHIP process is built on a more complex model. (Figure 
3) Following the CHIP model, members of local programs look 
at several factors in relation to schooling related 
activities: parent' involvement; federal, state and 
district level program requirements; and community 
contextual variables. This model recognizes that Chapter 1, 
regular classroom, and other programmatic instruction take 
place in more or less skillfully coordinated school 
settings. School achievement gain (and other desired 
outcomes) are presented in the CHIP model as being directly 
affected by parents. Chapter l instruction, regular 
classroom instruction, and other school programs (e.g.^ 
special education, migrant, bilingual, and gifted and 
talented programs) , and indirectly affected by school 
district^ state^ and federal policies and practices, and by 
the larger community. It is a built-in assumption that 
children learn from all of their school, family, and 
community interactions. 
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FIguw 3: More Realistic Compensatory Education Program 
Effects Modal 
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There are two sets of implications of this model for CHIP. 
The first concerns efforts aimed at improving student 
outcomes. ^ In the CHIP model, it is assumed that no single 
inteirvention is likely to produce improvement in all 
contexts. If, for example, current chapter 1 instruction is 
producing very high raters of Academic Learning Time (ALT) 
(Fisher et al*, 1979) during chapter 1 classes, an Effective 
Use of Time Program (Stallings, 1980), focused on chapter 1 
instructors, would be unlikely to produce improvement in 
student achievement. 

Instead of prescribing any single intervention, the CHIP 
model assumes that greatest gain can be achieved by 
allocating scarce resources to strengthening the weakest 
units or connections in the model. At one school or 
district, this might involve school-wide efforts to improve 
coordination among regular, chapter l, special Education, 
and Bilingual programr In an adjacent school or district, 
it could lead to effotx-j to enhance parents' involvement in 
their children's reading* 

Context dictates choices within CHIP* Before improvement 
activities are undertaken, projects must gather data to 
ascertain their current condition, both within the chapter 1 
program, and within the larger educational setting. 
Improvement efforts are directed toward improving the 
weakest perceived aspects of the model. 

A second set of implications stemming from the model concern 
measurement. The great majority of chapter 1 programs have 
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historically gathered "signal evaluation" data (King and 
Pechinan, 1984) to satisfy state and federal demands. This 
typically translated to aggregated participation, cost, and 
"Model A" (Pre-Post gain measured in NCEs on norm referenced 
tests) achievement measurement. Given the generally unmet 
psychometric concerns over the measurement of gain (Bryk and 
Weisberg, 1977; Rogosa, Brand and Zimowski, 1982; Sorenson 
and Halliman, 1977: Bryk and Raudenbush, 1988), "Unit of 
analysis" (Cronbach, 1976; Haney, 1980; Burstein, 1980), and 
measurement issues caused by the nesting of Chapter 1 
programs within and across schools, the value of such data 
is questionable. 

In order to have a more complete study of learning, the 
following are required: 

1) a model for individual growth, 

2) data from at least three points in time, 

3) analytic tools which accurately reflect the levels 

of the model. 

CHIP meets the model specification requirement, and 
sustained effects studies begin to address the "three points 
in time" condition. 

THE CHIP WORKSHOP SERIES 

The Chapter 1 program improvement workshop series posits a 
ten step process for planning and implementing change. 
Briefly, the ten steps are as follows: 

1. Identify key individuals to coordinate improvement. 

2. Become familiar with the research base regarding 
effective practices. 

3. Assess current practices. 

4. Target certain aspects of Chapter 1 practices for 
special emphasise. 

5. Set performance goals for targeted Chapter 1 
practices. 

6. Identify effective techniques to accomplish goals. 

7. Develop a plan. 

8. Implement a plan. 

9. Monitor progress and provide assistance. 
10. Reassess practices and renew efforts. 

The key themes underlying the CHIP are as follows: 

1. DATA DRIVEN: Participants are required to gather and to 
examine data about their programs. These data focus on two 
areas: current student outcomes as reflected by student 
achievement scores on norm referenced achievement tests and 
results of a self-assessment instrument, administered to 
administrators, regular classroom teachers, and Chapter 1 
staff in the participants' districts. These data yield a 
picture of the program at the onset of the program 
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improvement process, and serve two functions: 1) as a 
comparison point with which to quantify progress made and 2) 
as a system which allows reinforcement of the strengths of 
the program and targeting of areas which reflect the 
relative weaknesses or areas in which improvement efforts 
will be targeted. 

2. RESEARCH-BASED: Participants in the program are 
introduced and encouraged to use the research on chapter 1 
instruction and programs at all times. From the first 
worckshop where participants are given hands-on exercises 
utilizing the research through the planning effort and to 
implementation, participants are guided through information 
on effective practices and encouraged not to "re-invent the 
wheel", but rather to learn from what is known. 

3. TEAM CENTERED: When districts participate in CHIP, they 
are asked to do so in a team. The team is to be composed of 
the Chapter l coordinator, at least one principal, one 
regular classroom teacher, and one Chapter 1 teacher. 
Generally the chapter 1 coordinator heads the team. All 
team members are required, in so far as possible, to attend 
all of the workshops in the series and to complete all 
assignments. The latter usually involves meetings with 
school personnel anc", each other between workshops. The team 
approach is necessary to engender commitment, motivation, 
and responsiveness to the task. 

4. LOCAL CONTROL: The CHIP process is built on facilitating 
local decision-making. Using the data gathered from their 
own programs and the research base, the team of people from 
the district are encouraged to make programmatic decisions 
which best reflect the realities of their districts. They 
look at their particular situations with regard to 
personnel, politics, community support and the like, and 
plan changes that best fit the context in which they are 
operating. 

5. SYSTEMATIC PROCESSES: Throughout their participation, 
team members are trained in the use of systematic approaches 
to program improvement. Although there is great diversity 
in the content of the program improvement .plans, the process 
used by all participants to target areas for improvement, 
set performance goals, identify techniques to accomplish 
goals, develop plans, implement plans and monitor success is 
the same. 

6. FOCUS ON STUDENT OUTCOMES: Participants are encouraged 
to keep focused on the student learning. While 
instructional practices may be changed or organizational 
goals may be addressed, the key is always to examine how the 
students will be affected. 
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Workshop l of the CHIP assists participants in obtaining 
overviews of relevant "effectiveness" research and of thsir 
own programs relative to those research literatures, it 
begins with an overview of the wbrkshop series and the 10 
step process for planning and implementing change. Thirteen 
"Characteristics of Effective Compensatory Education 
Programs" as identified through a review of research by the 
U.S. Department of Education (Griswold, et.al., 1986) are 
introduced. These are as follows: 

o clear goals and objectives 

o Coordination with the regular school 

program/ other special programs 
o Parent/comnianity involvement 
o Professional development and- training 
o Strong leadership 

o Appropriate instructional materials, methods 

and approaches 
o High expectations for student learning and 

behavior 

o closely monitored student progress 
o Regular feedback and reinforcement 
o Excellence recognized and rewarded 
o Positive school/classroom climate 
o Maximum use of academic learning time 
o Evaluation results used for program or project 
improvement 

Highly interactive discussions of these characteristics are 
structured into the day. A more formal tool, "The Chapter 1 
Self Assessment Instrument" (Murray, Davis, and Stringfield, 
1986, 1988) is then introduced so that participants may 
systematically examine their own projects. A number of 
other tools, designed to evaluate various aspects of 
particpipants' local programs are presented. These 
facilitate districts'"' examination of student, school, and 
program level performance in the areas of instructional 
balance. Academic Learning Time, and similar topics. 

Between sessions one and two, participants are asked to 
perform self-assessment and systematic classroom 
observations to identify both the strengths and weaknesses 
of their current practices. Suggested readings are from 
primary and secondary research on instructional variables 
affecting student achievement. 

Workshop 2 enables participants to focus their improvement 
efforts. The workshop begins with presentations of data 
gained by local programs, including descriptions of the 
aggregated results of Self-Assessment, as completed by 
chapter 1 and other teachers, principals, and other involved 
administrators. Areas for improvement are targeted, and 
participants set performance goals and delineate strategies 
and techniques to achieve their goals. Additional 
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information is presented on specific effective practices. 
Examples are given of exemplary programs and practices from 
across the nation. Volumes 2 and 3 of the Effective 
Compensatory PHn n ation sourcebook (NWREL, 1986, 1987), and 
NWREL's "In their own words" series are central to this 
process. Specific topics addressed are determined to a 
large extent by outcomes of the self-assessments performed 
by the participants. As an assignment to be completed 
before the next session, participants are asked to develop 
specific objectives and strategies for change in one 
targeted area. A series of change planning and management 
forms and instructions are provided for that purpose. 

WorKshop _3 revolves around implementation and monitoring 
processes. Presentations focus on factors affecting the 
change process, and research based methods for change 
facilitation. Attention is paid to each district's 
particular change readiness, and strategies for working 
withm that situation. Participants are taught specific 
methods for monitoring and evaluating the effects of their 
efforts, and are reminded of the eventual need to renew the 
process. 

Specific issues addressed include staff development and 
program leadership, other research based topics may also be 
discussed. Technical assistance is offered in refining, 
implementing, and monitoring the plans. 

Changes in the series occurred as the research fields had 
undergone refinement, and as TAC staff expanded their 
experience. For example, the 1987-1988 series included 
refined sessions on leadership (e.g. Greenfield, 1986; 
Murphy and Hallinger, 1987; Bennis and Nanus, 1985; Kanter, 
1983); "higher order thinking skills" (e.g. Marzano et al. , 
1988; Resnick, 1987), and hands-on experiences with 
microcomputer based Chapter 1 data-bases which facilitate 
data use for targeted program improvement (e.g. Deck and 
Davis, 1988.) 

Throughout the series, the TAC and the local State 
Department offered on-site and telephone consultations. 
Materials were provided to assist participants in becoming 
familiar with the research base, to profile thei; 3wn 
programs, to plan improvements and to monitor outcomes. 
Ongoing technical assistance in subsequent improvement 
cycles was provided on request. 

EVALUATION OF THE CHIP SERTg S 

In the winter of 1989, an evaluation of the impact of the 
Chapter l program improvement process was conducted. The 
methods used to conduct this evaluation are presented next, 
followed by a discussion of results. 
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Methods 

Interviews were conducted with 35 participants from 33 
school districts in three of the states where the workshop 
was offered. Of these, 14 were from Colorado, 12 were from 
New Mexico, and 7 were from Wyoming. Districts represented 
rural, siiburban, and urban districts. 

The interviews used an open-ended format, asking questions 
and probes addressing the following areas: 

o changes, if any, made in the Chapter 1 program as a 
direct result of having participated in the program 
improvement workshop series 

o the process of implementing their improvement plans, 
any difficulties encountered 

o student outcomes as a result of the program improve- 
ment process 

o teacher outcomes as a result o."? the CHIP process 

o negatives associated with the process or with the 
changes 

o significant facilitators or barriers to change in the 
district 

o suggestions for improvement in the CHIP series 

Participants were asked to send data reflecting student 
achievement for the year immediately preceding their 
attendance in the series, the year that they attended the 
series, and the year after they attended, if they were 1986- 
1987 participants. 

Respondents were also asked to submit self-assessment data 
for the year of their participation and the year immediately 
following. If the latter did not exist, participants wera 
asked to re-administer the self-assessment and send the 
data. ^ Although qualitative data were gathered for all 33 
districts, quantitative data to date were less forthcoming. 
Fewer than a third of the districts have sent these data. 
While efforts are continuing to gather quantitative data, 
the analysis here will be confined to the information 
elicited in the interviews. 

Results 

Global responses to the follow up interviews ranged from 
"You ought to see the difference! We moved from an average 
graph (i.e, midrange scores on the self -assessment tool) all 
the way to the right (extremely positive scores). I'^^e 
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expanded the process to al3 of my other programs. It's the 
best thing in the world. If anyone follows through on it, 
they can't help being a success." (quote from a 
participant in a small, rural school district in New ff'^xico) 
to "Basically v;e didn't implement a thing." (quote from a 
participant in a rural district in Wyoming) 

Approximately two-thirds of those surveyed (63%) had changed 
their programs in a self -reported positive way, and 
attributed those changes directly to their participation in 
the workshop series. All respondents reported finding their 
participation positive, in the sense that they found the 
workshops informative and enjoyable. 

The effects of participation were extremely varied, ranging 
from total institutionalization of the CHIP process and of 
the planned efforts to no impact at all. These degrees of 
success are addressed next. 

Institutionalization of the cniF process 

Eight of the 33 districts sur-veyed have institutionalized 
CHIP in their districts. They require staff to perform 
self -assessments , become familiar with the research base, 
and use the CHIP planning process to organize change 
efforts. In some of the smaller districts, the approach has 
been mandated for all federal programs. In other districts, 
the particpipating teachers '^resented workshops throughout 
the district paralleling the series they were attending. 

The larger districts which institutionalized the process, 
did so in a way that increased their sense of ownership. In 
both of these examples, the districts changed the self- 
assessment tool to a shorter, more locally sensitive 
measurement. 

Other districts adopted the process, but redesigned the 
forms. Some added requirements to the process, such as 
individualized educational plans or staff development 
surveys. These "refinements" were encouraged, and most 
districts who report having made such changes also report 
the series as having very positive outcomes, particularly 
with regard to teachers. 

Institutionalization of the planned change 

Seventy-four percent of those surveyed reported implementing 
their plans for change made during the workshop. All of 
those who chose instructional areas report continuing 
success in their endeavors and positive outcomes for 
teachers. When respondents reported being able to link 
student outcomes to their change efforts, the changes they 
had made were typically in instructional areas. 
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One of the districts which targeted academic learning time, 
for example, required all Chapter 1 teachers to monitor and 
be monitored by each other. Each teacher was offered 
concrete suggestions and each instituted positive changes. 
When monitored again, each showed significant declines in 
nonengaged time in their classrooms. 

In one Wyoming district, the team decided to change their 
instructional approach. They reviewed the research, talked 
extensively with Chapter l staff in other programs, attended 
a reading conference, and visited other programs. This 
year y they initiated the new program, and report very 
positive teacher outcomes as well as anticipating very 
positive student outcomes based on observations and teacher 
judgments that has occurred thus far. According to these 
respondents, this action is directly attributable to the 
workshop series. 

One participant who was the sole representative from his 
district targeted rewards and reinforcement for student 
learning. He instituted a certificate program for good work 
awarded to students and congratulatory phone calls to 
parents. As the respondent put it, "the students feel good 
about themselves now, and I feel better becuuse they feel 
good" . 

Institutionalization of change in organizational areas was 
less clear. Those who targeted staff development, for 
example, reported positive change if they were given the 
freedom to design the type of staff development they wanted, 
and the opportunity to follow- through. Two of the cases 
were instructive in this regard: in one, the participants 
targeted and planned an extensive staff development effort, 
only to have the superintendent deny all requests for 
inseirvices that involved taking teachers out of the 
classroom. The team attempted to work around this rigidity, 
but reportedly were able to implement very little. 

In another case, the type of staff development was mandated 
from the top rather than from the team. Because the staff 
felt the effort was "top down" rather than "bottom-up", the 
effort reportedly failed. Staff attended the inservices, 
but made no changes in the classroom. In this case, 
however y other changes did reportedly occur in parent 
involvement, another area targeted by the team. This 
effort y though y has also foundered since the coordinator who 
lead the effort has been ill, and no one has assumed a 
strong leadership role. 

Those who target coordination between Chapter 1 and the 
regular classroom similarly report mixed results. An 
example of "success" here was reported by a team from New 
Mexico who changed the entire chapter 1 setting from pull- 
out to in-class. Success in coordinating between the 



14 



regular classroom and Chapter l, but not in the coordination 
of special education and Chapter l was reported by 
respondents in a district in Colorado. This success was 
attributed to having hired a teacher for Cnapter l who had 
attended the workshop series and who now used half of her 
time informally coordinating instruction between the school, 
grades, and classrooms. Several others reported mixed 
results, apparently dependent on either the principals' role 
m reinforcing the coordination or the presence of a 
reported "anti-Chapter 1 bias" among classroom teachers. 

The last point to be addressed with regard to targeting 
attributes for change revolves around the targeting of 
parent and community involvement. The majority of all of 
the participants from the school districts attending the 
workshop targeted parent involvement. There are several 
possible explanations for this: 1) this may be an artifact 
of the self -assessment instrument used (see Davis and 
Billig, 1989); 2) this may be the least threatening 
attribute for teams to target in the sense that none of 
those on the team or on the staff hold themselves 
responsible for parent involvement deficiencies; 3) this may 
be the weakest areas of these programs. 

Whatever the explanation, those who targeted parent 
involvement as a means of program improvement reported 
decidedly mixed success. That is, if success was defined as 
effort and implementation of plans, nearly all reported 
success. But if success was measured by outcome, i.e., 
increased parental involvement, many reported falling far 
short of their goals. Two cases are illustrative: in a 
large rural district in New Mexico, parent surveys vere 
distributed and collected, programs were designed to meet 
reported needs, the media were involved in advertising, and 
on the day of the first event, only 12 parents were present. 
The project had never had more than six parents attend 
previous meetings, so that in one sense, 12 participants 
represented success. Yet the staff had hoped for a turnout 
many times larger. The coordinator felt their hours of 
effort had not paid off. 

In a second case, in a suburban district in Colorado, a 
myriad of parent involvement activities occurred and the 
percentage of parents participating in the school, and 
working with their children at home increased dramatically, 
but when the self-assessment instrument was re-administered 
the following year, parent involvement again emerged as the 
area most in need of improvement. The second year data had 
a stifling effect on the team's willingness to engage in the 
CHIP process the next year. 



ERLC 



15 



Participating districts which reported no impact 

Six of the districts in this study reported attempting 
and/or institutionalizing little or no change as a result of 
their participation. Reasons cited for this generally 
reflected lack of leadership or lack of unity among the 
team. ^ One district participant from Wyoming reported 
enjoying the series but doing absolutely nothing with it. 
In this respondent's words, "There was no one to call (for) 
a meeting or to follow-up with." Another from a district in 
New Mexico said, "We had one teacher attending from each of 
the schools. They pretty much did their own plans. I was 
going to coordinate with them, but I ended up being switched 
back to the classroom, and I simply didn't have the time." 

In general, the districts which reported no impact also 
reported a lack of leadership, a lack of commitment to the 
process, and, in some cases, a conviction that change was 
not possible given their local circumstances. 

Teacher outcomes 

The majority of participants in the CHIP process reported 
very positive outcomes for teachers. Comments here 
indicated that teachers felt energized and sensitized as a 
result of their participation. Many felt that the 
presentation of exemplary characteristics served a valuable 
informative function in that teachers saw that a) success 
can be had with chapter 1 students; b) success can result 
from a variety of approaches; and c) change is possible. 

Teachers also reported value in the networking that took 
place. They felt they had access to new people, new ideas, 
and different programs. This empowered them in that avenues 
for seeking and finding information had opened for them, 
support systems for change existed, and expectations that 
change efforts would be made were normative. Many of the 
teams reported that there was increased collaboration among 
Chapter 1 teachers across schools and with others in their" 
buildings. In one New Mexico district, the team holds 
regular reunions to discuss the latest research in Chapter 
1. In a midsized Colorado district, the coordinator 
expressed great satisfaction with the new unity and pride 
felt by her Chapter 1 teachers as a result of their 
participation in the series. 

The strongest effects were reported by participants from 
rural, relatively isolated chapter 1 programs. Particularly 
when the coordinator was also the chapter 1 teacher, or when 
that person was the en^,ire chapter 1 staff, the reports of 
positive outcomes on the participant were very strong, and 
the sense of empowerment was pervasive. This was the case 
in six of the districts that were surveyed. 
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Teacher outcomes were strongest in those programs which 
chose instructional areas to target. However, this only 
occurred if the teachers were themselves committed to the 
process, in one district, for example, the teachers formed 
subcommittees to increase coordination, to investigate 
cooperative learning, and to survey their colleagues on the 
quality of staff development. The team met periodically to 
oversee these subcommittees and make recommendations for 
policy. 

In another district, the teachers who participated 
determined that they simply could not make any changes at 
all, that their case was hopeless, that they had no district 
or school support, and therefore it was not worth their time 
or effort to tri'. The only reported change that transpired 
in this program was the substitution of Apple computers for 
Hoffman machines: this change reportedly made the teachers 
unhappy. 

Student Outcomes 

Student outcomes have been much harder to attribute directly 
to participation in the CHIP process. Where achievement 
data are available (20% percent of the respondents made 
these data available), the direction of change was generally 
positive. However, too few districts reported this to be 
meaningful . 

Self-reports on student outcomes were also positive but 
these reports are suspect in that with probes, few could say 
that the relationship was direct. Much more information is 
needed here to draw conclusions. 

Negative outcomes Associated with CHIP 

The only negative outcomes reported were in two areas: the 
amount of paperwork and data collection required and 
negative evaluations of the self -assessment instrument. The 
former complaint, made by two districts, was in both cases 
tempered by a stated recognition of the necessity of the 
work. The comments regarding the self -assessment instrument 
used in the CHIP motivated the two districts making the 
complaints to develop their own self-assessment instruments, 
both of which are currently being used annually in these 
districts . 

Facilitators/Barriers to change 

This was an area respondents frequently had difficulty 
articulating. Most school districts, 21 out of 33, (63%) 
report no problem whatsoever in the planning or 
implementation process. These programs judged themselves to 
be successful with regard to all phases of CHIP, and were 
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the ones which institutionalized either the process, the 
plans, or both. 

Those sites with mixed outcomes or with participants who 
were not able to cite any outcomes from having participated 
provide interesting insight. A district in New Mexico, for 
example, sent a large team with the Chapter 1 coordinator 
and teachers from various schools to the CHIP series. The 
effect, as reported by the participants, was to make a good 
program even better, so much so that they applied for and 
won the Secretary's Recognition award the next year. During 
that year, the Chapter l coordinator required participation 
of all of the Chapter 1 teachers in the one school that, in 
his judgment, was not doing well. He attended along with 
these teachers, but the teachers alone targeted areas and 
drew up plans for change. The teachers never committed 
themselves to these plans, however, apparently because they 
felt they were there to please the coordinator. The result 
was no change that second year, but a major one this year: 
the coordinator was appointed Assistant Superintendent, and 
he has declared that school a schoolwide project. All of 
the school's staff are now involved in change efforts. 



Conclusions 

The nature of qualitative studies is such that every story 
is different: different circumstances, different players, 
different motives and means. The cases were filled with 
humanity: change grinding to a halt due to a pregnancy or 
the death of a spouse, districts attending because fathers 
were authorities at the state level, and so on. 

Yet through the diversity, common threads were established. 
In this section, conclusions that have emerged as a result 
of the interviews conducted will be presented. 

!• The CHIP process is an effective facilitator of change, 
but it is not a panacea. CHIP served as an effective 
program improvement process for the majority of participants 
surveyed. Changes were instituted. In many cases, the 
changes were reported to have had significant impact on the 
programs, and the CHIP process itself has been 
institutionalized. 

In evaluating CHIP as a process, it is clear that 
respondents generally saw value in their participation. 
Most participants planfully instituted change and were able 
to see the results of their efforts. Many reported 
significant positive teacher outcomes, and many could cite 
self -assessment data indicating positive change. However, 
there are not enough data in this study to know whether 
these changes have had a positive impact on student 
achievement . 

I'S 
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2. There are strong indications that certain factors are 
keys to success in bringing about change in Chapter 1 
programs. This study suggests that the following are of 
critical importance: 

a. Strong Leadership: As many researchers have found, 
strong leadership is a critical variable. The leader could 
be, and most often was, the Chapter 1 coordinator. 
Principals or teachers could also provide leadership 
although this study indicates that they may meet with less 
success. Competent leaders in organizations, and no less so 
m Chapter 1, hc^d important functions to serve in bringing 
about Changs: they need to manage attention, raaning and 
trust. 

Management of attention would implied that the leaders, 
perhaps along with the team, envision change and are able to 
compel commitment in others because of their focus. They 
have a vision in the most pragmatic sense: a set of 
intentions, goals and outcomes. The CHIP enabled the 
leaders and leadership teams to establish such a vision, to 
bring focus to the vision, and to approach that vision 
through the use of a systematic process. 

Leaders also managed meaning. Successful team leaders made 
their ideas tangible and salient so that their constituents 
could support them. This requires more than explanation or 
clarification. Rather, the leader or leadership team 
defined the goals and actively socialized others into 
believirg in them. Those involved came to be able to 
visualize the change and the means to achieve it. 
Successful CHIP teams were able to oroduce this through 
their leadership: the Chapter 1 staff, regular classroom 
teachers and sometimes parents came to view their programs 
differently, and this process was empowering. 

Management of trust implies constancy. Effective teams came 
together, expressing unity in action and thought. With 
effective leadership, they knew they could take risks, and 
could initiate change efforts without negative 
repercussions. Again, this was empowering. 

The flip side of this was lack of leadership. There is a 
term m medicine- iatrogenic- for illnesses caused by 
doctors or hospitals. There should be an analogous term in 
education foir deficiencies or barriers to improvement caused 
by administrators, teachers or schools, clearly this study 
indicates that lack of leadership or, as discussed below, 
lack of district support, can and occasionally did scuttle 
any change effort. 

b. District Support: This was' shown more to be a key when 
absent than when present, i.e., when there was district 
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support, it was frequently taken for granted. When it was 
absent, the changes simply were more difficult or iiapossible 
to bring about* In the case where leadership came in the 
form of the chapter 1 coordinator, district support tended 
to be more forthcoming. Such administrators had more 
influence over principals and could pave the way for teacher 
change and for such efforts as those surrounding 
coordination between the Chapter 1 program and the regular 
classroom. With district support, resources in the form of 
release time, consultants, new materials, and the like were 
also more likely to be available. These increased 
likelihood of successful change. 

c. Teacher ownership of the process. So-called "buy-in" is 
<pritical for success. Where participants felt changes were 
initiated top down with little attrmpt to involve them in 
deciding the character of the change or where participants 
attended the workshop because they felt they were required 
to do so, change was much less likely to occur. Where 
participants worked as a team to decide the direction and 
content of the change, the process was likely to be 
successful. 

d« Areas targeted for change: Change is a complex social 
process, it requires motivation and perceived benefit. 
What takes place in schools is affected by many variables: 
role relations among staff, patterns of communication, 
competitive domains of influence and sources of funding, 
capabilities of various individuals at every level in the 
hierarchy, the social and economic structuring of *:he school 
and the community, the philosophy of instruction f .id whether 
or not there is consensus on the approach: in short schools 
represent complex social systems within a complex social 
environment. It is within this system that Chapter 1 
operates, and within this system that Chapter 1 programs 
attempt to change. 

Within this context, it is easy to see why targeting 
instructional variables is most liKely to ba associated with 
success. ^ Changes in academic learning time, high 
expectations for student learning, feedback, monitoring 
students and rewarding excellence are more easily achieved. 
These areas are most directly controlled by Chapter 1 staff, 
and are most likely to be unaffected by other components in 
the educational social system. 

Similarly, organizational change which is confined to the 
program, i.e., staff development, positive climate, clear 
goals and objectives or using evaluation results for program 
improvement, is more likely to meet with success. In these 
areas, change typically can be brought about through 
individual or Chapter group effort rather than involving 
others within the system, though exceptions have been 
previously cited here* 

ERiC 2 0 



Those areas that tended to be associated with less success, 
I.e., coordination with the regular program and 
parent/ community involvement, involve changing others' 
behaviors and/or attitudes. This is indeed a more difficult 
and challenging task, one which cannot be totally determined 
oy Chapter i staff efforts klone. 

Among the projects sur^/eyed, great efforts to change tha 
Chapter l approach to parents were made, along with numerous 
attempts to provide more avenues for parent involvement. 
While respondents reported an increase in parent 
involvement, the effect typically was not viewed as 
sufficient to encourage further effort. Similarly, with 
regard to coordination, results were mixed. A key to 
coordination was the district's support. 



e. Importance of information dissemination and expectations 
fcr change in a cross-district group setting: The provision 
of networking, exemplary program information, and norms for 
change served a powerfully motivating function for some 
Chapter l program participants. Many reported being 
empowered by the information and this sense of being able to 
ma..e a difference, along with perceived access to technical 
assistance "experts" promoted very active efforts to plan 
and implement positive change;. 

Implications of this study for the progreun improvement 
provisions in tho chapter i reauthorization law 

P.L. 100-297, the law reauthorizing Chanter l, contains a 
number of program improvement provisions. Any chapter 1 
program which does not show "adequate" gains or which does 
not show aggregate gains by instructional area computed on a 
schoolwide basis is subject to targeting for program 
improvement. As a result of this targeting, a local 
education agency (LEA) must file a program improvement plan 
developed in consultation with parents and approved by the 
loca], Board of Education. If the program is agp.in targeted 
for improvement using the same criteria a second year, the 
Chapter 1 office of the state department of education (SEA) 
must join the LEA in developing local improvement plans. 

SEAs are currently in the process of enacting state plans 
for program improvement processes, and many are using 
approaches which closely parallel the one presented here. 
Several are using the self-assessment tool developed and 
used m CHIP, the same research base incorporating the 13 
attributes of exemplary compensatory education, and a 
planning approach which closely resembles this one. How can 
the results of this study be used as guidance for those 
about to embark on a similar course? 
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Many of the conclusions of this study are pertinent to the 
SEA plans. For example, it was found that where meaningful 
progr?m improvement efforts were made, LEAs generated 
commitment to the proposed change from those who would later 
implement it. External incentives and mandates were not 
enough. This issue will most likel>f be of even greater 
concern if the districts are not engaging in this process 
voluntarily. 

Commitment to a cbp.nge process can be elicited through 
various means. This study suggects that one approach may be 
to invite Chapter 1 and regular classroom teachers to be on 
the improvement team, and encouraging all of them to play an 
active role. By implication if parent or community 
involvement areas are to be targeted, representatives from 
those groups must also be actively recruited onto the 
improvement team. 

The teams should be lead by the Chapter 1 coordinator or 
another leader who could provide focus, obtain district 
support, and motivate the team members. Attention, meaning 
and trust should be managed. Having the administrative 
staff play an active role should also allow for more 
realistic planning in that these people have more control 
over resources and staff development activities. 

This study also found that the program improvement teams 
should target, in so far as possible, areas that are more 
likely to be responsive to change efforts. To sustain the 
improvement effort, partici'pants need to feel a sense of 
efficacy. This means the instructional areas and those 
organizational areas most under the direct influence of the 
Chapter 1 program should perhaps be the initial targets of 
change efforts. 

The process should be data driven. Individuals need to 
quantify their impact, and using self-assessments in 
addition to student and teacher outcomes will both give the 
process momentum and focus. The improvement teams will be 
able to identify and address the weakest links in their 
programs. In addition, effects will be measurable. 

The approach to be used in the program improvement process 
should be systematic. The new law requires timelinei^. and 
LEAs should use them judiciously. Planned change should be 
realistic, but ambitious. A systematic, plunful approach 
complete with assignments of responsibility, techniques tc 
accomplish goals, and built-in monitoring can work well. 

Finally, the district and state administrators who are 
involved in the program improvement effort should play an 
active role, not just have a symbolic presence. At the very 
least, it is vital that those engaging in the change process 
feel that the district will support their efforts. If 
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district administrators participate in the process at the 
outset, they can be involved in the direction of the change 
and can mediate between various interest groups and factions 
by presenting information on the larger district picture, 
and can steer team members into choosing areas that have the 
greatest probability for success. 

SUGGESTIONS FOR FUTURE RESEARCH 

This study has produced findings that indicate promise for 
the Chapter 1 ImprovemtTit Process as a tool for effective 
change in Chapter 1 programs, it is based on research, uses 
approaches which have produced results in a variety of 
contex::s, and is universally reported by participants to be 
en3oyable and informative. 

These findings, however, are suggestive rather than 
definitive. More data, especially on student outcomes, are 
needed to evaluata the impact of the process. In the 
future, it is suggested that measxirements be made of student 
achievement, program self -assessment, administrator and 
teacher perceptions and attitudes, and perhaps additional 
measurements of other aspects of the Chapter 1 program by 
various a/fected or affiliated individuals prior to the 
provision of the workshop series and again re-administered 
at various points after the intervention has taken place. 
This study is instructive in its analysis of trenc's, but for 
s.dded value, a more formal, systematic evaluation is needed. 

There are also promising directions for future study 
suggested by this research. Outcomes for volunteer versus 
nonvolunteer subjects is one such area to be investigated. 
As Rosenthal and Rosnow (1975) have pointed out, 
nonyolunteers are likely to be less enthusiastic and less 
motivated to change. Given the new program improvement 
provisions in the Chapter 1 reauthorization, this factor is 
likely to be salient, and should be investigated further. 

The influence of context is another promising area for 
research. Investigators should look at outcomes of the 
program improvement processes by urbanity and other social 
and economic variables to see whether they are producing 
differential effects. This research suggests that large 
urban schools benefit from the CHIP process in different 
ways than small rural ones. Given new rural education 
initiatives in Chapter 1, this area for research assumes 
added importance. 

Another area for research concerns the thirteen 
characteristics of exemplary chapter l programs and their 
relationship to student achievement. Several participants 
in the CHIP observed that these characteristics did not 
appear to have equal probabilities of influencing student 
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outcomes, studies connecting change efforts in these 
thirteen target, areas to student achievement are needed. 

Finally, the role of the Chapter 1 Technical Assistance 
Centers (TAC) in mediating program improvement efforts 
requires investigation. This study demonstrated that TACs 
can be effective change agents. The CHIP was offered by TAC 
staff in all of the cases presented here. These staff 
members come to the process with their own areas of 
expertise, access to information, and abilities to motivate 
participants to engage in change efforts. These variables 
are pertinent to the program improvement processes in that 
participants may be encouraged, though not necessarily 
consciously, to pursue certain types of change or to 
investigate certain practices more vigorously. It may also 
be that varying levels of expertise of the presenters of the 
workshop could influence the success of the series as a 
whole. 

Chapter 1 is at a critical juncture in its history with 
regard to program improvement, As programs attempt to 
comply with the new legislative mandates, evidence from the 
current study gives reason for optimism. 



ERIC 



24 



REgERENGES 



Barr, R. and Dreeben, R. (1983) . How schools Work . 
Chicago: University of Chicago Press. 

Bennis, W. and Nanus, B. (1985.) Leaders: The Strategies 
for Taking Charge. New York: Harper and Row. 

Brookover, W. , Beady, c. , Flood, P., Schweitzer, J., and 
Wisenbaker, j. (1979) . Schools social systems and 
student ach ievement: schools can make a difference. 
New York: Praeger. 

Brophy, J. (1981). Teacher praise: A functional analysis. 
Review of Educational Research. 51 . 5-32. 

Brophy, J. and Good, T. (1986). "Teacher Behavior and 
Student Achievement." In M. Wittrock, Handbook of 
Researc h on Teaching. Third Edition . New York: 
Macmillan. 

Clark, T. and McCarthy, D. (1983). School improvement in 
New York City: The evolution of a project. 
Educati onal Researcher ,. 12 (4), 17-24. 

Davis, A., (1988), "Educational Change and Local Context: 
Chapter i Program Improvement in Three Districts", 
paper presented at the American Educational Research 
Association Annual Meeting. 

Davis, A., and Blllig, S., (1989). "The Chapter 1 self- 
Assessment Instrument: The Internal Consistency of a 
Program Improvement Tool", presented at the American 
Educational Research Association Annual Meeting. 

Davis, A., and Deck, D. (1988). A microcomputer based 
Chapter 1 database management system. Denver: 
Northwest Regional Educational Laboratory.. 

Drucker, P. (1966) . The Effective Executive. New York: 
Harper cdid Row. 

Drucker, p. (1985) . Innovation and entrepreneurship. New 
York: Harper and Row. 

Edmonds, R. (1979) . Effective schools for the urban poor. 
Educational Leadership ^ 37, 15-24.* 



25 



Fisher, C. , Berliner, D. , Filby, N. , Marliave, R., Cahen, 
L. , and Dishaw, M. (1980). "Teaching behaviors, 
academic learning time, and student achievement: An 
overview." In C. Denham and A. Lieberman (Eds.), Time 
to learn. Washington, D.C.: National Institute of 
Education. 

Fullan, M. (19S2) . The meaning of educational change. New 
York: Teachers College Press. 

Gage, N. (1985) . Hard gains in the soft Sciences: The c. se 
of pedagogy. Bloomington, Ind. : Phi Delta Kappa. 

Good, T. and Brophy, J. (1986). "School Effects." In M. 
Wittrock, Handbook of Research on Tpaching. Third 
Edition . New York: Macmillan. 

Good, T. , and Grouws, D. (1979). The Missouri Mathematics 

Effectiveness Project: An experimental study in fourth 
grade classrooms. Journal of Educational Psychology. 
Zl, 355-362. 

Greenfield, W. (ed.) (1987). Instructional Leadership. 
Newton, Mass.: Allyn and Bacon. 

Griswold, P., Cotton, K., and Hansen, j. (1986). Effective 
Compensatory Education Sourcebook . Washington, D.C.: 
U.S. Government Printing Office. ~ 

Hall, G., and Hord, S. (1987). Change in schools: 

Facilitation the process. Albany: SUNY Press. 

Huberman, M. , and Miles, h. (1984). Innovation up close: 
How sch ool improvement works. New York: Plenum. 

Kanter, r. (1983). The change masters. New York: Simon 
and Schuster. 

King, J. and Pechman, E. (1984). Pinning the wave to the 
shore: Conceptualizing school evaluation use. 
Educati onal Evaluation and Policy Analysis . 6,3, 241- 
251. 

Leitnsr, David, (1988) . "Wi thin-District Common and 

Divergent Features of Involvement in a Chapter 1 
Program Improvement Project", paper presented at the 
American Educational Research Association annual 
meeting. 

Levin, H. , ^ix.v, Msisi:er, . (1986). "Is CAI cost effective?" 
Phi Delta Kappan . 67, 10, 745-749. 



ERIC 



26 



Lewis, B,, (ed.) (1987) • "In their own words: Effective 
Chapter 1 programs as described by their staffs," 
Denver: Northwest Regional Educational Laboratory 

Lieberman, A. (ed.) (1986). Rethinking school improvement: 
Researc h, craft, and concept . New York: Teachers 
College Press. 

Marzano, R. , Brandt, R. , Hughes, C, Jones, B. , Presseisen, 
B., Rankin, S., Suhor, C. (1988). Dimensions of 
thinkin g; A framework for curriculum and instruction. 
Alexandria, Va.: Association for Supervision and 
Curriculum Development. 

Murphy, J. and Hallinger, P. (eds. ) (1987) . Approaches to 
adminis trative training in education. Albany: State 
University of New York Press. 

Murray, S. Davis, A., and Stringfield, S. (1986, 1988). 
Chapter l Self Assessment Instrument". Denver, 
Northwest Regional Educational Laboratory. 

Peters, T. and Austin, N. (1985.) A Passion for Excellence. 
New York: Random House. 

Purkey, S. and Smith, M. (1983). Effective Schools: A 
review. Elementary School Journal . 81, 427-452. 

Resnick, L. (1987). Education and learning to think. 
Washington, D.C.: National Academy Press. 

Rosenblum, S., and Seashore-Louis, K. (1981). Stability and 
Change: Innovation in an educational context. New 
York: Plenum. 

Rowan B. and Denk, C.E. (1982). Modeling the academic 

performance of schools using longitudinal data: As 
analysis of school effectiveness measures and school 
and Pri ncipal effects on school level achievement . San 
Francisco: Far West Lab. 

Rowan, B., Guthrie, L. , Lee, G. and Guthrie, G. , (1986). The 
Design and Implementation of Chapter 1 Instructional 
Services : A Study of 24 Schools ^ San Francisco: Far 
West Laboratories 

Showers, B, , Joyce, B. , and Bennett, B. (1987). Synthesis 
of research on staff development: A framework for 
future study end a state-of-the-art analysis. 
Educati onal Leadership ^ 45 , 3, 77-87. 

Stallings, j. (1980). Academic learning time revisited, or 
beyond time on task. Educational Researcher . 8. 11 , 
11-16. ■ 

ERIC 2 7 



27 



Stringfield, S., Schaffer, E. and Devlin-Scherer, R. (1986). 
A study of the generalizability of findings from 
teacher effectiveness experiments. Journal of 
Classroom Interaction , 21, 2, 9-15. 

Stringfield, S. and Teddlie, C. (October, 1988). A time to 
sumnarize: six years and three phases of the Louisiana 
School Effectiveness Study. Educational Leadership , 
46f 1 (in press) . 

Teddlie, C. , Falkowski, Stringfield, S., Desselle, S. 

and Garvue, r. (1984). The Louisiana School 
Effectiveness Studv: Phase II 1982-1984 . Baton Rouge, 
LA: The Louisiana State Department of Education. 

Wimpelberg, R. , Teddlie, C. and Stringfield, S. (1989)1 
Sensitivity to context in school effectiveness 
research. Educational Administration Quarterly , 25, 1 
(in press) . 



ERiC 



28 



